Image processing apparatus and control method for the same

ABSTRACT

A direction in which pixels highly correlated with a signal value of a pixel of interest exist is determined by using both signal values of a plurality of pixels of different colors from the pixel of interest and of a plurality of pixels of the same color as the pixel of interest among a plurality of peripheral pixels of the pixel of interest, and obtains pixel correction values corresponding to the respective directions. Weighted addition of those pixel correction values is performed in accordance with the degree to which the pixel of interest is of an achromatic color, and the pixel correction value for the pixel of interest is thus obtained.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus and acontrol method therefor, and, more particularly, relates to an imageprocessing apparatus for processing a captured image and a controlmethod therefor.

2. Description of the Related Art

A solid-state image sensor such as a CCD has several million or morepixels, and it is therefore difficult to eliminate pixels from whichnormal output signals cannot be obtained (defective pixels). However,since output signals from the defective pixels cannot be used as theyare, defective pixel correction, in which predicted values obtained fromoutput signals of peripheral normal pixels are used as the outputsignals of those defective pixels, is performed (Japanese PatentLaid-Open No. 11-220661).

Although various methods for obtaining a predicted value (pixelcorrection value) from output signals of peripheral normal pixels areknown, a typical one will be described here. FIG. 11A is a diagramshowing a part of the pixels in an image sensor having a so-calledBayer-pattern primary color filter, in which red (R) pixels, green (G)pixels, and blue (B) pixels are regularly arranged; the red pixel R33 atthe center is assumed to be a defective pixel.

First, absolute differences are found between the output values of pixelpairs located in the vertical direction, the horizontal direction, the45-degree direction, and the 135-degree direction respectively relativeto the defective pixel among the peripheral pixels (B22, G23, B24, G34,B44, G43, B42, and G32) of the defective pixel. Specifically, supposethe absolute difference for the horizontal direction is H_DIV, theabsolute difference for the vertical direction is V_DIV, the absolutedifference for the 45-degree direction is D45_DIV, and the absolutedifference for the 135-degree direction is D135_DIV. Then, the followingare calculated:H_DIV=ABS(G32−G34)V_DIV=ABS(G23−G43)D45_DIV=ABS(B24−B42)D135_DIV=ABS(B22−B44)Then, the direction of the smallest absolute difference is determined tobe the direction in which pixels highly correlated with the pixel ofinterest exist. An average value of the output values of a pair of thepixels of the same color as the defective pixel, which are located tosandwich the defective pixel in the determined direction, is used as apixel correction value for the defective pixel.

That is, suppose the pixel correction value is Pix, and then:

-   -   If H_DIV is the smallest: Pix=(R31+R35)/2    -   If V_DIV is the smallest: Pix=(R13+R53)/2    -   If D45_DIV is the smallest: Pix=(R15+R51)/2    -   If D135_DIV is the smallest: Pix=(R11+R55)/2

However, in this method, the direction determination is performed byusing only the signal values of the eight peripheral pixels surroundingthe pixel of interest. Therefore, a problem arises in that if anyinfluence from noise or the like is contained in the signal values ofthe eight peripheral pixels, an error may occur in directiondetermination, and as a result, the accuracy of pixel correction valuesis reduced.

SUMMARY OF THE INVENTION

The present invention was made in consideration of such problems foundin the conventional technique, and provides an image processingapparatus capable of more stably obtaining accurate directiondetermination results, and a control method therefor.

According to the first aspect of the present invention, there isprovided an image processing apparatus for generating a signal value ofa pixel of interest in an image, which is captured by an image sensorhaving a color filter, from signal values of a plurality of peripheralpixels of the pixel of interest, comprising: a determination unitadapted to output a determination signal indicating a degree to whichthe pixel of interest is of an achromatic color based on a chroma signalcorresponding to the pixel of interest; an achromatic color directiondetermination unit adapted to determine a direction, in which a pixelhighly correlated with the signal value of the pixel of interest exists,from the signal values of a plurality of pixels of a different colorfrom the pixel of interest among the plurality of peripheral pixels ofthe pixel of interest; a first correction value generation unit adaptedto generate the signal value of the pixel of interest by using a signalvalue of a pixel that exists in the direction determined by theachromatic color direction determination unit among the plurality ofperipheral pixels of the pixel of interest; a chromatic color directiondetermination unit adapted to determine a direction, in which a pixelhighly correlated with the signal value of the pixel of interest exists,from signal values of a plurality of pixels of the same color as thepixel of interest and signal values of a plurality of pixels of adifferent color from the pixel of interest among the plurality ofperipheral pixels of the pixel of interest; a second correction valuegeneration unit adapted to generate the signal value of the pixel ofinterest by using a signal value of a pixel that exists in the directiondetermined by the chromatic color direction determination unit among theplurality of peripheral pixels of the pixel of interest; and a weightedaddition unit adapted to perform weighted addition of the signal valuegenerated by the first correction value generation unit and the signalvalue generated by the second correction value generation unit such thata weight of the signal value generated by the first correction valuegeneration unit is larger and a weight of the signal value generated bythe second correction value generation unit is smaller as the degree towhich the pixel of interest is of an achromatic color indicated by thedetermination signal is higher, and calculating a final signal value ofthe pixel of interest.

According to the second aspect of the present invention, there isprovided an image capture apparatus comprising: an image sensor having acolor filter; and the image processing apparatus according to the firstaspect of the present invention, wherein in an image captured by theimage sensor, a signal value of a defective pixel in the image sensor isacquired by the image processing apparatus.

According to the third aspect of the present invention, there isprovided an image capture apparatus comprising: an image sensor having acolor filter; and the image processing apparatus according to the firstaspect of the present invention, wherein the signal value calculated bythe weighted addition unit in the image processing apparatus is used incolor interpolation processing on a pixel of interest in an imagecaptured by the image sensor.

According to the fourth aspect of the present invention, there isprovided a method for controlling an image processing apparatus forgenerating a signal value of a pixel of interest in an image, which iscaptured by an image sensor having a color filter, from signal values ofa plurality of peripheral pixels of the pixel of interest, the methodcomprising: a determination step of outputting a determination signalindicating a degree to which the pixel of interest is of an achromaticcolor based on a chroma signal corresponding to the pixel of interest;an achromatic color direction determination step of determining adirection, in which a pixel highly correlated with the signal value ofthe pixel of interest exists, from the signal values of a plurality ofpixels of a different color from the pixel of interest among theplurality of peripheral pixels of the pixel of interest, a firstcorrection value generation step of generating the signal value of thepixel of interest by using a signal value of a pixel that exists in thedirection determined in the achromatic color direction determinationstep among the plurality of peripheral pixels of the pixel of interest,a chromatic color direction determination step of determining adirection, in which a pixel highly correlated with the signal value ofthe pixel of interest exists, from signal values of a plurality ofpixels of the same color as the pixel of interest and signal values of aplurality of pixels of a different color from the pixel of interestamong the plurality of peripheral pixels of the pixel of interest; asecond correction value generation step of generating the signal valueof the pixel of interest by using a signal value of a pixel that existsin the direction determined in the chromatic color directiondetermination step among the plurality of peripheral pixels of the pixelof interest; and a weighted addition step of performing weightedaddition of the signal value generated in the first correction valuegeneration step and the signal value generated in the second correctionvalue generation step such that a weight of the signal value generatedin the first correction value generation step is larger and a weight ofthe signal value generated in the second correction value generationstep is smaller as the degree to which the pixel of interest is of anchromatic color indicated by the determination signal is higher, andcalculating a final signal value of the pixel of interest.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments, with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an exemplary functional configurationof an image capture apparatus serving as an exemplary apparatus to whichan image processing apparatus according to the first embodiment of thepresent invention can be applied;

FIG. 2 is a flowchart for explaining the operation of a chromatic colordetermination circuit in FIG. 1;

FIG. 3 is a diagram showing an exemplary relationship among a value of achroma signal C, the thresholds Th1 and Th2, and a value of a monochromedetermination signal D be in the first embodiment of the presentinvention;

FIG. 4 is a block diagram showing an exemplary functional configurationof an achromatic color pixel correction circuit in FIG. 1;

FIG. 5 is a flowchart for explaining the operation of the achromaticcolor pixel correction circuit in FIG. 1;

FIG. 6 is a block diagram showing an exemplary functional configurationof a chromatic color pixel correction circuit in FIG. 1;

FIG. 7 is a flowchart for explaining the operation of ahorizontal/vertical-direction possibility determination circuit in FIG.6;

FIG. 8 is a flowchart for explaining the operation of an HV pixelcorrection circuit in FIG. 6;

FIG. 9 is a flowchart for explaining the operation of an obliquecorrection circuit in FIG. 6;

FIG. 10 is a diagram for explaining aliasing effect caused by directiondetermination using the pixels of the same color as a pixel of interest;

FIGS. 11A to 11E are diagrams for explaining direction determinationprocessing and pixel correction value generation processing according tothe first embodiment of the present invention; and

FIG. 12 is a block diagram showing an exemplary functional configurationof an image capture apparatus serving as an example of an apparatus towhich an image processing apparatus according to the second embodimentof the present invention can be applied.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the present invention will now be described indetail in accordance with the accompanying drawings.

First Embodiment

FIG. 1 is a block diagram showing an exemplary functional configurationof an image capture apparatus serving as an exemplary apparatus to whichan image processing apparatus according to an embodiment of the presentinvention can be applied. Note that FIG. 1 shows, among components ofthe image capture apparatus, only the components associated withdefective pixel correction. Also, the components related to imagecapturing are not essential in the image processing apparatus, accordingto the present invention.

Note that the functional block of a pixel correction circuit 104described below may be implemented by computer executing software or byhardware logic. The image processing apparatus according to the presentembodiment may be implemented by, for example, a CPU executing a controlprogram to control hardware such as a storage device, or an interfaceprovided in a general computer.

In FIG. 1, an image capture unit 101 has a photographing lens, an imagesensor, and a drive circuit therefor (which are not shown in thediagram); the image capture unit 101 converts an optical image formed bythe photographing lens into an electric signal with the image sensor. ACCD image sensor or a CMOS image sensor, for example, is used as theimage sensor, which is provided with a color filter in which a pluralityof specific colors are arranged regularly. Here, it is assumed that aBayer pattern primary color filter shown in FIG. 11A is used as anexample of such a color filter. In FIG. 11A, Rxx denotes a red pixel,Gxx denotes a green pixel, and Bxx denotes a blue pixel (xx is anumber).

An analog image signal for each pixel output from the image capture unit101 is converted by an A/D conversion circuit 102 into a digital imagesignal, and input to a white balance (WB) circuit 103. The WB circuit103 performs known white balance adjustment on the digital image signal.An image signal output from the WB circuit 103 is input to a pixelcorrection circuit 104, which corresponds to an image processingapparatus according to the present embodiment.

The pixel correction circuit 104 has an achromatic color pixelcorrection circuit 110, a chromatic color pixel correction circuit 111,a chromatic color determination circuit 112, and a weighted additioncircuit 113. The pixel correction circuit 104 obtains a pixel correctionvalue, which is a predicted value of an image signal of a defectivepixel, based on position information on the defective pixel.Specifically, output values of the achromatic color pixel correctioncircuit 110 and the chromatic color pixel correction circuit 111 aresubjected to weighted addition in the weighted addition circuit 113according to the result of determination by the chromatic colordetermination circuit 112, and thus the pixel correction value isobtained. The pixel correction value obtained in the pixel correctioncircuit 104 is input to a signal processing circuit 105. The signalprocessing circuit 105 is provided with image signals of pixels otherthan the defective pixel and the pixel correction value, and performsknown color interpolation processing, luminance signal processing, colorsignal processing, and the like.

Note that in the present embodiment, the pixel correction circuit 104generates a pixel correction value if a pixel of interest (defectivepixel) is a red pixel (R pixel) or a blue pixel (B pixel). If the pixelof interest is a green pixel (G pixel), since the number of green pixels(G pixels) is large, direction determination and interpolation using theabove-described conventional method can be performed based only on theperipheral green pixels. Processing related to such pixel correctionvalue generation for G pixels is performed by using a correction circuitnot shown in the figure (alternatively, the pixel correction circuit 104may be used).

Note that the pixel correction circuit 104 may also be configured tooutput signals of pixels other than the defective pixel as they are tothe signal processing circuit 105, and, as for the defective pixel,output the obtained pixel correction value to the signal processingcircuit 105.

Note that the pixel correction circuit 104 is capable of acquiring, orcan acquire in advance, information on correspondence between inputimage signals and pixel arrangement on the image sensor, and theposition of the defective pixel. The method for implementing this is notparticularly limited, but, for example, information about the positionof the defective pixel in the image sensor included in the image captureunit 101 may be registered in the pixel correction circuit 104, anonvolatile memory, or the like, that can be accessed by the pixelcorrection circuit 104 at the time of manufacturing the image captureapparatus or the like. Furthermore, the pixel correction circuit 104 maybe notified of position information on the defective pixel via imagesignals. Also, the arrangement of pixel signals included in imagesignals input to the pixel correction circuit 104 is determined inadvance, and accordingly, peripheral pixels of a specific pixel, or thelike, can be specified. Further, it is assumed that information aboutthe arrangement and colors in the color filter is also known in advance.The configuration with which the pixel correction circuit 104 obtainsthose items is not directly related to the essence of the presentinvention, and arbitrary known methods may be employed; therefore, anyfurther description thereof will be omitted.

Operation of the Pixel Correction Circuit

Hereinafter, defective pixel correction processing in the pixelcorrection circuit 104 will be described. Note that in order to helpunderstanding of the present invention and simplify the description, itis assumed here that the pixel of interest (the defective pixel) is ared pixel (R pixel) or a blue pixel (B pixel); a case in which the pixelof interest is an R pixel will be described. However, in the case inwhich the pixel of interest is a B pixel, the same processing may alsobe performed by applying the pixel arrangement relationship as describedbelow.

Operation of the Color Determination Circuit

Firstly, the operation of the chromatic color determination circuit 112will be described with reference to the flowchart in FIG. 2. First, inS201, the chromatic color determination circuit 112 calculates a chromasignal C of an image signal after being subjected to white balanceprocessing at the pixel of interest according to, for example, thefollowing expressions;Y=0.3R+0.59G+0.11BCr=0.713(R−Y)Cb=0.564(B−Y)C=√(Cr ² +Cb ²)

The values of R, G, and B at the pixel of interest will be describedlater. Next, in S202, if the value of the chroma signal C is smallerthan the threshold Th1 at a certain level, the chromatic colordetermination circuit 112 determines that the pixel of interest is of anachromatic color and sets 1 for a monochrome determination signal D_bc(S203). Meanwhile, in S202, if the value of the chroma signal C is equalto or larger than the threshold Th1, the chromatic color determinationcircuit 112 advances the processing to S204 and compares the value ofthe chroma signal C with the prescribed threshold Th2 larger than thethreshold Th1. If the value of the chroma signal C is larger than thethreshold Th2, the chromatic color determination circuit 112 determinesthat the pixel of interest is of a chromatic color and sets 0 for themonochrome determination signal D_bc (S205). Meanwhile, in S204, if thevalue of the chroma signal C is equal to or smaller than the thresholdTh2, a value obtained by linear interpolation in accordance with thevalue of the chroma signal C is set for the monochrome determinationsignal D_bc (S206). FIG. 3 shows an exemplary relationship among thevalue of the chroma signal C, the thresholds Th1 and Th2, and the valueof the monochrome determination signal D_bc.

As described above, the chromatic color determination circuit 112outputs the monochrome determination signal D_bc of a value “1” if thevalue of the chroma signal C is smaller than the threshold Th1, andoutputs the monochrome determination signal D_bc of a value “0” if thevalue of the chroma signal C is larger than the threshold Th2. Also, ifthe value of the chroma signal C is in the range from the threshold Th1to the threshold Th2, the chromatic color determination circuit 112outputs the monochrome determination signal D_bc of a value expressed byD_bc=1−(C−Th1)/(Th2−Th1). Thus, the value of the monochromedetermination signal D_bc can be considered as indicating a degree towhich the pixel of interest is of an achromatic color.

Next, an example of methods for obtaining the R, G, and B values at thepixel of interest to obtain the chroma signal C will be described. FIG.11B shows a pixel arrangement associated with part of an image signalafter being subjected to white balance processing. Here, the red pixelR33 is assumed to be the pixel of interest (i.e., the defective pixel),and R, G, and B at pixel of interest R33 are obtained by using signalvalues at peripheral pixels of the pixel of interest.

In the present embodiment, the peripheral pixels used to obtain thechroma signal C at the position of the pixel R33 are G21, G41, G25, andG45.

First, the chromatic color determination circuit 112 obtains missingvalues among the R, G, and B values at the peripheral pixels from thesignal values of adjacent pixels. Here, because all peripheral pixelsare green pixels, the R and B values are obtained from the signal valuesof adjacent red and blue pixels.

Suppose the R and B values at the position of the pixel G21 are R_21 andB_21, respectively, and then, the chromatic color determination circuit112 obtains those values as follows:R _(—)21=(R11+R31)/2B _(—)21=(B20+B22)/2

Similarly, the chromatic color determination circuit 112 also obtainsthe R and B values at the pixels positions of G41, G25, and G45 asfollows:R _(—)41=(R31+R51)/2B _(—)41=(B40+B42)/2R25=(R15+R35)/2B _(—)25=(B24+B26)/2R _(—)45=(R35+R55)/2B _(—)45=(B44+B46)/2

Next, the chromatic color determination circuit 112 obtains therespective average values of the R, G, B values at the pixel positionsof G21, G41, G25, and G45 as the R, G, and B values at pixel of interestR33. That is, the following expressions are applicable:R _(—)33=(R _(—)21+R _(—)41+R _(—)25+R _(—)45)/4G _(—)33=(G21+G41+G25+G45)/4B _(—)33=(B _(—)21+B _(—)41+B _(—)25+B _(—)45)/4

Note that in the present embodiment the peripheral pixels used to obtainthe chroma signal C at the position of the pixel R33 are G21, G41, G25,and G45, but a pixel group at other positions may alternatively be used.Further, a configuration in which saturation is normalized based onbrightness so as to also detect the saturation in a dark area may beemployed. For example, the saturation can be normalized based onbrightness by dividing the saturation by luminance Y.

Operation of the Weighted Addition Circuit

Before describing the operation of the achromatic color pixel correctioncircuit 110 and the chromatic color pixel correction circuit 111, theoperation of the weighted addition circuit 113 will be described.Suppose the pixel correction value output by the achromatic color pixelcorrection circuit 110 is Pix_1, and the pixel correction value outputby the chromatic color pixel correction circuit 111 is Pix_2. Theweighted addition circuit 113 obtains the final pixel correction valuePix in accordance with the value of the monochrome determination signalD_bc output by the chromatic color determination circuit 112, asfollows:Pix=(1−D _(—) bc)*Pix_(—)1+D _(—) bc*Pix_(—)2

Then, the weighted addition circuit 113 outputs the pixel correctionvalue Pix as the image signal of the defective pixel to the signalprocessing circuit 105.

Achromatic Color Pixel Correction Circuit

FIG. 4 is a block diagram showing an exemplary functional configurationof the achromatic color pixel correction circuit 110. An achromaticcolor direction determination circuit 410 performs directiondetermination processing for determining the direction in which pixelshighly correlated with the pixel of interest (the defective pixel) exist(i.e., the direction in which pixels exist that should be used incalculation of the pixel correction value for the pixel of interest).

An achromatic color pixel correction value selection circuit 405selects, in accordance with the result of determination by theachromatic color direction determination circuit 410, one of the pixelcorrection values output by an average correction circuit 400, an H(0-degree) correction circuit 401, a V (90-degree) correction circuit402, a 45-degree correction circuit 403, and a 135-degree correctioncircuit 404.

Next, the operation of the achromatic color pixel correction circuit 110will be described using the flowchart in FIG. 5. In the directiondetermination in the present embodiment, pixel pairs respectivelylocated in the vertical (90-degree) direction, the horizontal (0-degree)direction, the 45-degree direction, and the 135-degree directionrelative to the pixel of interest at the center in the pixel arrangementshown in FIG. 11C are selected, and the direction of the selected pixelpair that has the smallest absolute difference between the signal valuesis the determination result.

In S501, the achromatic color direction determination circuit 410 firstcalculates the absolute differences between the signal values of pairsof the peripheral pixels located so as to sandwich the pixel of interestin a specific direction (the vertical direction, the horizontaldirection, the 45-degree direction, or the 135-degree direction). Here,as shown in FIG. 11C, R11, G12, R13, G14, R15, G21, G25, R31, R35, G41,G45, R51, G52, R53, G54, and R55 are used as the peripheral pixels ofthe pixel of interest (defective pixel) R33. Next, the achromatic colordirection determination circuit 410 obtains the absolute differences forthe horizontal direction, the vertical direction, the 45-degreedirection, and the 135-degree direction with respect to each peripheralpixel as follows.

Suppose the absolute difference at the pixel R11 for the horizontaldirection is H_DIV_11, the absolute difference for the verticaldirection is V_DIV_11, the absolute difference for the 45-degreedirection is D45_DIV_11, and the absolute difference for the 135-degreedirection is D135_DIV_11, and then:H_DIV_(—)11=ABS(G10−G12)V_DIV_(—)11=ABS(G01−G21)D45_DIV_(—)11=ABS(B02−B20)D135_DIV_(—)11=ABS(B00−B22)

In this case, although H_DIV_11 and V_DIV_11 are obtained by using thesignal values of G pixels, and D45_DIV_11 and D135_DIV_11 are obtainedby using the signal values of B pixels, the direction determination canbe correctly performed. This is because, in the case of an achromaticcolor subject, the signal values of a G pixel and a B pixel are equal toeach other, and therefore both can be regarded as luminance signals.Thus the achromatic color direction determination circuit 410 performsthe direction determination by using the equivalent signal values ofpixels when considering the subject to be of an achromatic color, and isthereby able to obtain an accurate direction determination result if thesubject is actually of an achromatic color.

The achromatic color direction determination circuit 410 similarlyobtains the absolute differences for the four directions also withrespect to the other peripheral pixels G12, R13, G14, R15, G21, G25,R31, R35, G41, G45, R51, G52, R53, G54, and R55. Finally, the achromaticcolor direction determination circuit 410 obtains average values of theabsolute differences obtained with respect to the respective peripheralpixels for the respective directions.

That is, suppose the average values obtained for the respectivedirections are H_DIV_AVE, V_DIV_AVE, D45_DIV_AVE, and D135_DIV_AVE,respectively, and then:H_DIV_AVE=(H_DIV_(—)11+H_DIV_(—)12+ . . . +H_DIV_(—)55)/16V_DIV_AVE=(V_DIV_(—)11+V_DIV_(—)12+ . . . +V_DIV_(—)55)/16D45_DIV_AVE=(D45_DIV_(—)11+D45_DIV_(—)12+ . . . +D45_DIV_(—)55)/16D135_DIV_AVE=(D135_DIV_(—)11+D135_DIV_(—)12+ . . . +D135_DIV_(—)55)/16

In S502, the achromatic color direction determination circuit 410determines whether or not all average values for the respectivedirections H_DIV_AVE, V_DIV_AVE, D45_DIV_AVE, and D135_DIV_AVE of theabsolute differences at the peripheral pixels are equal to or smallerthan the threshold Th1. If all average values of the absolutedifferences for the respective directions are equal to or smaller thanthe threshold Th1, the achromatic color direction determination circuit410 determines that no specific direction exists in which pixels highlycorrelated with the pixel of interest exist. Then, the achromatic colordirection determination circuit 410 outputs a signal for selecting thepixel correction value obtained by the average correction circuit 400(S504).

The average correction circuit 400 calculates the pixel correctionvalue, Pix, for the pixel R33 as an average value of signal values ofthe pixels R13, R31, R35, and R53 of the same color as the pixel R33that appear next (i.e., that are adjacent) in the horizontal directionand the vertical direction, as follows:Pix=(R13+R31+R35+R53)/4

Meanwhile, in S502, if at least one of the average values of theabsolute differences for the respective directions exceeds the thresholdTh1, the achromatic color direction determination circuit 410 advancesthe processing to S503, and determines the direction of the smallestaverage value of the absolute differences as the direction in whichpixels highly correlated with the pixel of interest exists. Theachromatic color direction determination circuit 410 then outputs asignal for selecting the pixel correction value corresponding to thedetermined direction from among the pixel correction values output bythe H (0-degree) correction circuit 401, the V (90-degree) correctioncircuit 402, the 45-degree correction circuit 403, and the 135-degreecorrection circuit 404.

That is, the achromatic color direction determination circuit 410outputs, to the achromatic color pixel correction value selectioncircuit 405, signals respectively for selecting:

-   -   the pixel correction value output by the H (0-degree) correction        circuit 401 if H_DIV_AVE is the smallest;    -   the pixel correction value output by the V (90-degree)        correction circuit 402 if V_DIV_AVE is the smallest;    -   the pixel correction value output by the 45-degree correction        circuit 403 if D45_DIV_AVE is the smallest; and    -   the pixel correction value output by the 135-degree correction        circuit 404 if D135_DIV_AVE is the smallest.

The H (0-degree) correction circuit 401, the V (90-degree) correctioncircuit 402, the 45-degree correction circuit 403, and the 135-degreecorrection circuit 404 calculate pixel correction values for the pixelof interest from the signal values of pixels existing in a specificdirection relative to the pixel of interest at the center. Here, anaverage value of the signal values of two pixels of the same color asthe pixel of interest that exist in a reference direction with the pixelof interest therebetween is obtained as the pixel correction value.

Specifically, suppose that the pixel value after being subjected topixel correction is Pix, and then,

-   -   the H (0-degree) correction circuit 401 calculates        Pix=(R31+R35)/2;    -   the V (90-degree) correction circuit 402 calculates        Pix=(R13+R53)/2;    -   the 45-degree correction circuit 403 calculates Pix=(R15+R51)/2;        and    -   the 135-degree correction circuit 404 calculates Pix=(R11+R55)/2    -   as the achromatic color pixel correction values for the pixel        R33, respectively.

As described above, both signal values of a G pixel and a B pixel can beregarded as luminance signal values with respect to an achromatic colorsubject, and therefore the direction determination can be correctlyperformed. Further, the absolute differences for the respectivedirections at each of a plurality of peripheral pixels (16 in thepresent embodiment) of the pixel of interest are obtained, and thedirection determination is performed based on the average values of theabsolute differences for the respective directions; thus it is possibleto reduce erroneous direction determination affected by noise.

Note that in the present embodiment, as a method for obtaining the pixelcorrection value for the defective pixel, a method in which an averagevalue of signal values of the pixels of the same color as the defectivepixel that exist in the reference direction with the defective pixeltherebetween is calculated has been described. However, pixel correctionvalues obtained by any methods using signal values of pixels existing inthe reference direction from the defective pixel may be used. Similarly,the number and position of the peripheral pixels used in the directiondetermination are not limited to the above-described specific number andposition.

Chromatic Color Pixel Correction Circuit

FIG. 6 is a block diagram showing an exemplary functional configurationof the chromatic color pixel correction circuit 111. A chromatic colorpixel correction value selection circuit 622 selects one of the pixelcorrection value output by a vertical/horizontal pixel correctioncircuit 620 and the pixel correction value output by an oblique pixelcorrection circuit 621 in accordance with a result obtained by ahorizontal/vertical-direction possibility determination circuit 612, andoutputs the selected pixel correction value as the final chromatic colorpixel correction value.

The vertical/horizontal pixel correction circuit 620 has an HV directiondetermination circuit 610, an average correction circuit 600, an H(0-degree) correction circuit 601, a V (90-degree) correction circuit602, and an HV pixel correction value selection circuit 613. The obliquepixel correction circuit 621 has an oblique direction determinationcircuit 611, a 45-degree correction circuit 603, a 135-degree correctioncircuit 604, an oblique average correction circuit 605, and an obliquepixel correction value selection circuit 614. Note that in FIG. 7, theconfigurations of the average correction circuit 600, the H correctioncircuit 601, the V correction circuit 602, the 45-degree correctioncircuit 603, and the 135-degree correction circuit 604 may be the sameas those of the blocks 400 to 404 with the same names in FIG. 4, andtherefore a detailed description thereof will be omitted.

Horizontal/Vertical-Direction Possibility Determination Circuit

Firstly, the horizontal/vertical-direction possibility determinationcircuit 612 will be described. As shown in FIG. 11D, when the pixel ofinterest (the defective pixel) is R33, the horizontal/vertical-directionpossibility determination circuit 612 determines the horizontal orvertical-direction possibility from the signal values of the peripheralpixels R11, R13, R15, R31, R35, R51, R53, and R55 of the same color asthe pixel of interest. Specifically, the horizontal/vertical-directionpossibility determination circuit 612 obtains the absolute differencebetween the signal values of the peripheral pixels of the same colorexisting respectively in specific directions (here, the vertical,horizontal, 45-degree, and 135-degree directions) relative to the pixelof interest R33. Here, suppose the absolute difference for thehorizontal direction is H_DIV, the absolute difference for the verticaldirection is V_DIV, the absolute difference for the 45-degree directionis D45_DIV, and the absolute difference for the 135-degree direction isD135_DIV, and then, the horizontal/vertical-direction possibilitydetermination circuit 612 calculates:H_DIV=ABS(R31−R35)V_DIV=ABS(R13−R53)D45_DIV=ABS(R15−R51)D135_DIV=ABS(R11−R55)

Now the aliasing effect caused by the direction determination using thepixels of the same color as the pixel of interest will be described.FIG. 10 shows a spatial frequency domain, where fs indicates a samplingfrequency. Areas V1 and H1 within the Nyquist frequency (fs/2) are notaffected by aliasing, and are therefore correctly determined to be inthe V direction and the H direction respectively. However, areas V2 andV3, which are actually in the H direction, are determined to be in the Vdirection due to aliasing. Areas H2 are H3, which are actually in the Vdirection, are determined to be in the H direction also due to aliasing.Accordingly, in areas H2, H3, V2, and V3, the H (horizontal) directionand the V (vertical) direction cannot be correctly distinguished. In aBayer pattern color filter, the number of R pixels and B pixels are halfthe number of G pixels, and therefore, the vertical direction and thehorizontal direction cannot be distinguished based on the directiondetermination using R pixels (B pixels) only. Similarly, 45-degree and135-degree directions cannot be distinguished either. However, it ispossible to find whether the horizontal or vertical direction or theoblique direction based on the direction determination using R pixels (Bpixels) only.

Accordingly, the horizontal/vertical-direction possibility determinationcircuit 612 determines whether the direction in which pixels highlycorrelated with the pixel of interest exist is the horizontal orvertical direction or the oblique direction—that is, distinguishes thetype of direction—from the signal values of the peripheral pixels of thesame color as the pixel of interest.

Horizontal/vertical-direction possibility determination processing willbe described in accordance with the flowchart in FIG. 7. In 5701, thehorizontal/vertical-direction possibility determination circuit 612calculates the absolute difference H_DIV for the horizontal direction,the absolute difference V_DIV for the vertical direction, the absolutedifference D45_DIV for the 45-degree direction, and the absolutedifference D135_DIV for the 135-degree direction, respectively, asdescribed above. Next, in S702, the horizontal/vertical-directionpossibility determination circuit 612 determines whether the directionin which pixels highly correlated with the pixel of interest exist isthe horizontal or vertical direction or the oblique direction.Specifically, suppose the sum of the HV absolute differences is HV_DIVand the sum of the oblique absolute differences is D_DIV, and then, thehorizontal/vertical-direction possibility determination circuit 612obtains:HV_DIV=H_DIV+V_DIVD_DIV=D45_DIV+D135_DIV

Then, if HV_DIV is smaller than D_DIV, the horizontal/vertical-directionpossibility determination circuit 612 determines that pixels highlycorrelated with the pixel of interest exist in the horizontal orvertical direction, and outputs a horizontal/vertical-directionpossibility determination signal D_hv having a value “1” (S703).Meanwhile, if HV_DIV is equal to or larger than D_DIV, thehorizontal/vertical-direction possibility determination circuit 612determines that pixels highly correlated with the pixel of interestexist in the oblique direction, and outputs thehorizontal/vertical-direction possibility determination signal D_hvhaving a value “0” to the chromatic color pixel correction valueselection circuit 622 (S704).

The chromatic color pixel correction value selection circuit 622 selectsand outputs the pixel correction value output by the vertical/horizontalpixel correction circuit 620 if the value of thehorizontal/vertical-direction possibility determination signal D_hv is“1”, and selects and outputs the pixel correction value output by theoblique pixel correction circuit 621 if the value of thehorizontal/vertical-direction possibility determination signal D_hv is“0”.

Vertical/Horizontal Pixel Correction Circuit

Next, the operation of the vertical/horizontal pixel correction circuit620 will be described in accordance with the flowchart in FIG. 8. The HVdirection determination circuit 610 performs the direction determination(determination of whether it is the vertical direction or the horizontaldirection) using the signal values of green pixels.

In S801, the HV direction determination circuit 610 calculates theabsolute differences of the signal values of green pixels located in thevertical direction and the horizontal direction, with respect to each ofa plurality of peripheral pixels of the pixel of interest. For example,in the example shown in FIG. 11D, suppose the absolute differencebetween the signal values of the green pixels located in the horizontaldirection with respect to the peripheral pixel R11 is H_DIV_11, and theabsolute difference between the signal values of the green pixelslocated in the vertical direction is V_DIV_11, and then, the HVdirection determination circuit 610 calculates:H_DIV_(—)11=ABS(G10−G12)V_DIV_(—)11=ABS(G01−G21)

The HV direction determination circuit 610 similarly obtains theabsolute differences between the signal values of the green pixelslocated in the vertical direction and the horizontal direction withrespect each of to the peripheral pixels R13, R15, R31, R35, R51, R53,and R55 of the same color as the pixel of interest, and after that,calculates average values according to the respective directions.Suppose the average value for the horizontal direction is H_DIV_AVE, andthe average value for the vertical direction is V_DIV_AVE, the HVdirection determination circuit 610 calculates:H_DIV_AVE=(H_DIV_(—)11+H_DIV_(—)13+ . . . +H_DIV_(—)55)/8V_DIV_AVE=(V_DIV_(—)11+V_DIV_(—)13+ . . . +V_DIV_(—)55)/8

Next, in S802, the HV direction determination circuit 610 determineswhether or not both average values H_DIV_AVE and V_DIV_AVE of theabsolute differences for the respective directions calculated in S801are equal to or smaller than the threshold Th1. If both average valuesof the absolute differences for the respective directions are equal toor smaller than the threshold Th1, the HV direction determinationcircuit 610 determines that any specific direction in which pixelshighly correlated with the pixel of interest exist does not exist, andoutputs a signal for selecting the pixel correction value obtained bythe average correction circuit 600 (S804).

Meanwhile, in S802, if at least one of the average values of theabsolute differences for the respective directions exceeds the thresholdTh1, the HV direction determination circuit 610 advances the processingto S803, and determines the direction of the smaller average value ofthe absolute differences to be the direction in which pixels highlycorrelated with the pixel of interest exist. Then, the HV directiondetermination circuit 610 outputs a signal for selecting the pixelcorrection value corresponding to the determined direction from amongthe pixel correction values output by the H (0-degree) correctioncircuit 601 and the V (90-degree) correction circuit 602. Specifically,the HV direction determination circuit 610 outputs a signal forselecting the pixel correction value of the H (0-degree) correctioncircuit 601 if H_DIV_AVE is the smaller, and a signal for selecting thepixel correction value output by the V (90-degree) correction circuit602 if V_DIV_AVE is the smaller.

Oblique Correction Circuit

Next, the operation of the oblique pixel correction circuit 621 will bedescribed in accordance with the flowchart in FIG. 9. The obliquedirection determination circuit 611 also performs directiondetermination (however, determination of the 45-degree direction or the135-degree direction) using the signal values of green pixels. In S901,the oblique direction determination circuit 611 calculates the absolutedifferences between the signal values of green pixels located in the45-degree direction and the 135-degree direction with respect to aplurality of peripheral pixels of the pixel of interest. For example, inthe example shown in FIG. 11E, suppose the absolute difference betweenthe signal values of the green pixels located in the 45-degree directionwith respect to the peripheral pixel G12 is D45_DIV_12, and the absolutedifference between the signal values of the green pixels located in the135-degree direction is D135_DIV_12. In this case, the oblique directiondetermination circuit 611 calculates:D45_DIV_(—)12=ABS(G03−G21)D135_DIV_(—)12=ABS(G01−G23)

The oblique direction determination circuit 611 similarly obtains theabsolute differences between the signal values of the green pixelslocated in the 45-degree direction and 135-degree direction with respectto each of the other peripheral pixels G14, G21, G25, G41, G45, G52, andG54 shown in FIG. 11E, and after that, calculates the average values forthe respective directions. Suppose the average value for the 45-degreedirection is D45_DIV_AVE, and the average value for the 135-degreedirection is D135_DIV_AVE, and then, the oblique direction determinationcircuit 611 calculates:D45_DIV_AVE=(D45_DIV_(—)12+D45_DIV_(—)14+ . . . +D45_DIV_(—)54)/8D135_DIV_AVE=(D135_DIV_(—)12+D135_DIV_(—)14+ . . . +D135_DIV_(—)54)/8

Next, in S902, the oblique direction determination circuit 611determines whether both average values D45_DIV_AVE and D135_DIV_AVE ofthe absolute differences for the respective directions calculated inS901 are equal to or smaller than the threshold Th1. If both averagevalues of the absolute differences for the respective directions areequal to or smaller than the threshold Th1, the oblique directiondetermination circuit 611 determines that any specific direction inwhich pixels highly correlated with the pixel of interest exist does notexist, and outputs a signal for selecting the pixel correction valueobtained by the oblique average correction circuit 605 (S904).

The oblique average correction circuit 605 calculates the pixelcorrection value Pix for the pixel R33 as the average value of thesignal values of the pixels R15, R51, R11, and R55 of the same color asthe pixel R33 that appear next (i.e., that are adjacent) in the45-degree direction and the 135-degree vertical direction, as below:Pix=(R15+R51+R11+R55)/4

Meanwhile, in S902, if at least one of the average values of theabsolute differences for the respective directions exceeds the thresholdTh1, the oblique direction determination circuit 611 advances theprocessing to S903, and determines the direction of the smaller averagevalue of the absolute differences to be the direction in which pixelshighly correlated with the pixel of interest exist. Then, the obliquedirection determination circuit 611 outputs a signal for selecting thepixel correction value corresponding to the determined direction fromamong the pixel correction values output by the 45-degree correctioncircuit 603 and the 135-degree correction circuit 604. Specifically, theoblique direction determination circuit 611 outputs a signal forselecting the pixel correction value output by the 45-degree correctioncircuit 603 if D45_DIV_AVE is the smaller, and a signal for selectingthe pixel correction value output by the 135-degree correction circuit604 if D135_DIV_AVE is the smaller.

When determining the direction in which pixels highly correlated withthe pixel of interest exist, the signal values of the peripheral pixelsof the same color as the pixel of interest can be used, but accuratedirection determination cannot be always performed on a chromatic colorsubject due to problems of subject colors and aliasing. Therefore, inthe present embodiment, only the type of direction in which pixelshighly correlated with the pixel of interest exist is determined fromthe signal values of the peripheral pixels of the same color as thepixel of interest. The specific direction is determined by using thesignal values of the peripheral pixels of different colors from thepixel of interest.

For example, in an image captured by an image sensor having a Bayerpattern color filter, if an R (B) pixel is the pixel of interest(defective pixel), it is determined, from the signal values ofperipheral R (B) pixels, whether the direction in which pixels highlycorrelated with the pixel of interest exist is the horizontal orvertical direction or the oblique direction. Then, it is determined,from the signal values of peripheral G pixels of the pixel of interest,which specific direction the direction in which pixels highly correlatedwith the pixel of interest exist is.

Therefore, it is possible to reduce the possibility of erroneousdirection determination also with respect to a chromatic color subject.Further, even with respect to a subject whose G pixel signal level issmall, such as a red subject, it is correctly determined as to whetherit is the horizontal or vertical direction, or the oblique direction.Therefore, the possibility of erroneous direction determination usingthe signal values of G pixels due to noise can also be reduced.Accordingly, an accurate pixel correction value can be obtained byapplying the present invention to the defective pixel correction. Notethat the effect of accuracy improvement on direction determination on achromatic color subject is an effect that can be achieved solely by thechromatic color pixel correction circuit 111.

Note that the method for calculating a pixel correction value afterdirection determination is not directly related to the essence of thepresent invention, and arbitrary methods other than the one specificallydescribed above are available.

According to the present embodiment, it is determined whether thesubject of the pixel of interest is of achromatic colors or chromaticcolors, and if the subject is of achromatic colors, the directiondetermination is performed by using the signal values of the peripheralpixels, regardless of whether those peripheral pixels are of the samecolor as or a different color from the pixel of interest. Therefore, itis possible to refer to the signal values of a larger number ofperipheral pixels than before with respect to an achromatic colorsubject, and to thus obtain an accurate direction determination result.

Meanwhile, with respect to a chromatic color subject, the directiondetermination using the signal values of the peripheral pixels of thesame color as the pixel of interest, as well as the directiondetermination using the signal values of peripheral pixels of differentcolors from the pixel of interest are performed. Therefore, as describedabove, it is possible to perform more accurate direction determinationthan direction determination using only pixels of the same color as thepixel of interest or direction determination using only pixels ofdifferent colors from the pixel of interest.

Furthermore, if it is not clear whether a subject is a chromatic colorsubject or an achromatic color subject, weighted addition of a pixelcorrection value based on a direction determination result in the casewhere the subject is regarded as an achromatic color subject, and apixel correction value based on a direction determination result in thecase where the subject is regarded as a chromatic color subject, iscalculated in accordance with saturation. Therefore, erroneouscorrection based on incorrect direction determination can also bereduced.

Second Embodiment

The present invention can be applied to not only generation of a pixelcorrection value for a defective pixel, but also color interpolationprocessing on a pixel of interest.

FIG. 12 is a block diagram showing an exemplary functional configurationof an image capture apparatus in which an image processing apparatusaccording to the second embodiment of the present invention is appliedto color interpolation processing. In FIG. 12, an image capture unit1501, an A/D conversion circuit 1502, a WB circuit 1503, and a signalprocessing circuit 1505 have the same configurations as those in FIG. 1,respectively, and therefore the detailed description thereof will beomitted. A color interpolation circuit 1504 is constituted from anachromatic color interpolation circuit 1510, a chromatic colorinterpolation circuit 1511, a chromatic color determination circuit1512, and a weighted addition circuit 1513. Note that the chromaticcolor interpolation circuit 1511 may be a known color interpolationcircuit, and therefore a detailed description thereof will be omitted.Also, the chromatic color determination circuit 1512 and the weightedaddition circuit 1513 are the same as those described in the firstembodiment, and therefore the detailed description thereof will beomitted. The specific configuration of the achromatic colorinterpolation circuit 1510 is the same as that shown in FIG. 4, and itsoperation is also as described with reference to FIG. 5, and therefore adetailed description thereof will be omitted.

The difference from the first embodiment is in whether or not the pixelof interest is a defective pixel. In other words, in the presentembodiment the pixel of interest is a normal pixel, and so the signalvalue of the pixel of interest can be used in the directiondetermination.

In FIG. 11C, it is assumed that the pixel of interest is the pixel R33(which is not a defective pixel). Suppose an absolute difference for thehorizontal direction at the pixel R11 is H_DIV_11, an absolutedifference for the vertical direction is V_DIV_11, an absolutedifference for the 45-degree direction is D45_DIV_11, and an absolutedifference for the 135-degree direction is D135_DIV_11, and then:H_DIV_(—)11=ABS(G10−G12)V_DIV_(—)11=ABS(G01−G21)D45_DIV_(—)11=ABS(B02−B20)D135_DIV_(—)11=ABS(B00−B22)

In this case, H_DIV_11 and V_DIV_11 are obtained by using the signalvalues of G pixels, and D45_DIV_11 and D135_DIV_11 are obtained by usingthe signal values of B pixels, but the direction determination can becorrectly performed. This is because, in the case of an achromatic colorsubject, signal values of a G pixel and a B pixel are equal signalvalues to each other, and both can be regarded as luminance signals.

The achromatic color direction determination circuit 410 similarlyobtains absolute differences for the four directions also, with respectto other peripheral pixels. The other peripheral pixels are G12, R13,G14, R15, G21, B22, G23, B24, G25, R31, G32, R33, G34, R35, G41, B42,G43, B44, G45, R51, G52, R53, G54, and R55. Lastly, the achromatic colordirection determination circuit 410 calculates average values of theabsolute differences obtained with respect to the respective peripheralpixels for the respective direction.

In other words, suppose the average values for the respective directionsare H_DIV_AVE, V_DIV_AVE, D45_DIV_AVE, and D135_DIV_AVE, and then:H_DIV_AVE=(H_DIV_(—)11+H_DIV_(—)12+ . . . +H_DIV_(—)55)/25V_DIV_AVE=(V_DIV_(—)11+V_DIV_(—)12+. . . +V_DIV_(—)55)/25D45_DIV_AVE=(D45_DIV_(—)11+D45_DIV_(—)12+ . . . +D45_DIV_(—)55)/25D135_DIV_AVE=(D135_DIV_(—)11+D135_DIV_(—)12+ . . . +D135_DIV_(—)55)/25

Thus, compared with the first embodiment, in the present embodiment theabsolute differences are calculated also with respect to B22, G23, B24,G32, R33, G34, B42, G43, and B44, and the average values of 25 pixelsare obtained for the respective directions.

The direction determination processing may be performed similarly toS502 in the first embodiment. Thus, the direction determination by theachromatic color direction determination circuit 410 is more unlikely tobe affected by noise. It is possible to use the signal value of thepixel of interest, which cannot be used in the case of the defectivepixel correction described in the first embodiment, and therefore, thedirection determination can be more correctly performed by using thesignal values of pixels near the pixel of interest.

Also, after the direction determination, color interpolation may beperformed by using pixels of different colors from the pixel of interestthat exist in the determined direction. In this case, colorinterpolation may be performed in consideration of the signal values ofnot only the pixels existing in the determined direction, but also otherperipheral pixels. For example, weighted addition can be performed suchthat the weight of the signal values of the pixels peripheral to thepixel of interest that are of one color different in color from thepixel of interest and exist in the determined direction is larger thanthat of the signal values of the peripheral pixels of the same color notexisting in the determined direction.

Other Embodiment

In the above-described embodiment, if the pixel of interest is a Gpixel, for example, a pixel correction value may be generated by usingthe achromatic color pixel correction circuit 110. For example, in FIG.11D, if G23 is the pixel of interest (defective pixel), the achromaticcolor direction determination circuit 410 obtains:H_DIV=ABS(G21−G25)V_DIV=ABS(G03−G43)D45_DIV=ABS(G14−G32)D135_DIV=ABS(G12−G34)Then, the direction of the smallest absolute difference is determined tobe the direction in which pixels highly correlated with the pixel ofinterest exist.

Meanwhile, the H correction circuit 401 calculates the pixel correctionvalue of Pix=(G21+G25)/2, the V correction circuit 402 calculates thepixel correction value of Pix=(G03+G43)/2, the 45-degree correctioncircuit 403 calculates the pixel correction value of Pix=(G14+G32)/2,and the 135-degree correction circuit 404 calculates the pixelcorrection value of Pix=(G12+G34)/2.

The achromatic color direction determination circuit 410 outputs asignal for selecting the pixel correction value corresponding to thedetermined direction to the achromatic color pixel correction valueselection circuit 405.

In the above-described embodiment, the case of image data captured by animage sensor having a Bayer pattern color filter has been described.However, the principle of the present invention can be applied to imagedata captured by an image sensor having an color filter, in which aplurality of specific colors are regularly arranged, and the number ofpixels differs depending on color. In other words, the present inventioncan be applied when generating a pixel correction value for a pixel of acolor of which the number of pixels is small.

Further, in the above-described embodiment, the pixel correction values(including average correction values) for the respective directions thatcan be determined by the HV direction determination circuit 610 and theoblique direction determination circuit 611 are individually calculatedby the respective correction circuit. Then, a single pixel correctionvalue is selected in accordance with the result of directiondetermination by the HV direction determination circuit 610, and asingle pixel correction value is selected in accordance with the resultof direction determination by the oblique direction determinationcircuit 611.

However, a correction circuit capable of selectively performing thefunctions of the average correction circuit 600, the H correctioncircuit 601, and the V correction circuit 602 may be provided, and apixel correction value for the horizontal or vertical direction may begenerated based on the result of determination by the HV directiondetermination circuit 610. Similarly, a correction circuit capable ofselectively performing the functions of the 45-degree correction circuit603, the 135-degree correction circuit 604, and the oblique averagecorrection circuit 605 may be provided, and a pixel correction value inthe oblique direction may be generated based on the result ofdetermination by the oblique direction determination circuit 611. Inthose cases, the HV pixel correction value selection circuit 613 and theoblique pixel correction value selection circuit 614 are not necessary.

Further, a configuration in which one of the results of determination bythe HV direction determination circuit 610 and the oblique directiondetermination circuit 611 is selected based on the output of thehorizontal/vertical-direction possibility determination circuit 612 isalso available. In this case, only the correction value corresponding tothe selected determination result may be generated, and accordingly, thechromatic color pixel correction value selection circuit 622 is notnecessary either.

Basically, arbitrary configurations may be employed as long as one ofthe results of determination by the HV direction determination circuit610 and the oblique direction determination circuit 611 can be selectedin accordance with the result of determination of whether the horizontalor vertical direction or the oblique direction by thehorizontal/vertical-direction possibility determination circuit 612, andthe pixel correction value corresponding to the selected direction canbe generated.

Further, in the above-described embodiment, the direction determinationis performed for four directions, namely 0-degree, 45-degree, 90-degree,and 135-degree directions, but those directions may be changed dependingon the shape and arrangement of the color filter (pixels).

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a CPU or MPU) that reads out andexecutes a program recorded on a memory device to perform the functionsof the above-described embodiment (s), and by a method, the steps ofwhich are performed by a computer of a system or apparatus by, forexample, reading out and executing a program recorded on a memory deviceto perform the functions of the above-described embodiment (s). For thispurpose, the program is provided to the computer, for example via anetwork or from a recording medium of various types serving as a memorydevice (e.g., a computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2011-174254, filed on Aug. 9, 2011, which is hereby incorporated byreference herein its entirety.

What is claimed is:
 1. An image processing apparatus for generating asignal value of a pixel of interest in an image, which is captured by animage sensor having a color filter, from signal values of a plurality ofperipheral pixels of the pixel of interest, comprising: a determinationunit adapted to output a determination signal indicating a degree towhich the pixel of interest is of an achromatic color based on a chromasignal corresponding to the pixel of interest; an achromatic colordirection determination unit adapted to determine a direction, in whicha pixel highly correlated with the signal value of the pixel of interestexists, from the signal values of a plurality of pixels of a differentcolor from the pixel of interest among the plurality of peripheralpixels of the pixel of interest; a first correction value generationunit adapted to generate the signal value of the pixel of interest byusing a signal value of a pixel that exists in the direction determinedby the achromatic color direction determination unit among the pluralityof peripheral pixels of the pixel of interest; a chromatic colordirection determination unit adapted to determine a direction, in whicha pixel highly correlated with the signal value of the pixel of interestexists, from signal values of a plurality of pixels of the same color asthe pixel of interest and signal values of a plurality of pixels of adifferent color from the pixel of interest among the plurality ofperipheral pixels of the pixel of interest; a second correction valuegeneration unit adapted to generate the signal value of the pixel ofinterest by using a signal value of a pixel that exists in the directiondetermined by the chromatic color direction determination unit among theplurality of peripheral pixels of the pixel of interest; and a weightedaddition unit adapted to perform weighted addition of the signal valuegenerated by the first correction value generation unit and the signalvalue generated by the second correction value generation unit such thata weight of the signal value generated by the first correction valuegeneration unit is larger and a weight of the signal value generated bythe second correction value generation unit is smaller as the degree towhich the pixel of interest is of an achromatic color indicated by thedetermination signal is higher, and calculating a final signal value ofthe pixel of interest.
 2. The image processing apparatus according toclaim 1, wherein the achromatic color direction determination unitobtains, for each of the peripheral pixels of the pixel of interest, adifference between signal values of two pixels that are of a differentcolor from the pixel of interest and located so as to sandwich theperipheral pixel for a horizontal direction, a vertical direction, and aplurality of predetermined oblique directions, respectively, obtains anaverage value of the differences for each of the directions, anddetermines a direction corresponding to a smallest value of the averagevalues obtained for the respective directions to be the direction inwhich a pixel highly correlated with the signal value of the pixel ofinterest exists.
 3. The image processing apparatus according to claim 1,wherein the chromatic color direction determination unit comprises:first direction determination unit adapted to determining whether adirection in which a pixel highly correlated with the signal value ofthe pixel of interest exists is a horizontal or vertical direction or anoblique direction, from signal values of the plurality of peripheralpixels of the pixel of interest that are of the same color as the pixelof interest; second direction determination unit adapted to determining,as the direction in which a pixel highly correlated with the signalvalue of the pixel of interest exists, one of the vertical andhorizontal directions and one of a plurality of predetermined obliquedirections, from the signal values of the plurality of peripheral pixelsof the pixel of interest that are of a different color of the pixel ofinterest; and selection unit adapted to selecting the one of thevertical and horizontal direction determined by the second directiondetermination unit if the direction in which a pixel highly correlatedwith the signal value of the pixel of interest exists is determined tobe the vertical or horizontal direction by the first directiondetermination unit, and selects the one of the plurality of obliquedirections determined by the second direction determination unit if thedirection in which a pixel highly correlated with the signal value ofthe pixel of interest exists is determined to be the oblique directionby the first direction determination unit.
 4. The image processingapparatus according to claim 2, wherein the plurality of predeterminedoblique directions are a 45-degree direction and a 135-degree direction.5. An image capture apparatus comprising: an image sensor having a colorfilter; and the image processing apparatus according to claim 1, whereinin an image captured by the image sensor, a signal value of a defectivepixel in the image sensor is acquired by the image processing apparatus.6. An image capture apparatus comprising: an image sensor having a colorfilter; and the image processing apparatus according to claim 1, whereinthe signal value calculated by the weighted addition unit in the imageprocessing apparatus is used in color interpolation processing on apixel of interest in an image captured by the image sensor.
 7. A methodfor controlling an image processing apparatus for generating a signalvalue of a pixel of interest in an image, which is captured by an imagesensor having a color filter, from signal values of a plurality ofperipheral pixels of the pixel of interest, the method comprising: adetermination step of outputting a determination signal indicating adegree to which the pixel of interest is of an achromatic color based ona chroma signal corresponding to the pixel of interest; an achromaticcolor direction determination step of determining a direction, in whicha pixel highly correlated with the signal value of the pixel of interestexists, from the signal values of a plurality of pixels of a differentcolor from the pixel of interest among the plurality of peripheralpixels of the pixel of interest, a first correction value generationstep of generating the signal value of the pixel of interest by using asignal value of a pixel that exists in the direction determined in theachromatic color direction determination step among the plurality ofperipheral pixels of the pixel of interest, a chromatic color directiondetermination step of determining a direction, in which a pixel highlycorrelated with the signal value of the pixel of interest exists, fromsignal values of a plurality of pixels of the same color as the pixel ofinterest and signal values of a plurality of pixels of a different colorfrom the pixel of interest among the plurality of peripheral pixels ofthe pixel of interest; a second correction value generation step ofgenerating the signal value of the pixel of interest by using a signalvalue of a pixel that exists in the direction determined in thechromatic color direction determination step among the plurality ofperipheral pixels of the pixel of interest; and a weighted addition stepof performing weighted addition of the signal value generated in thefirst correction value generation step and the signal value generated inthe second correction value generation step such that a weight of thesignal value generated in the first correction value generation step islarger and a weight of the signal value generated in the secondcorrection value generation step is smaller as the degree to which thepixel of interest is of an chromatic color indicated by thedetermination signal is higher, and calculating a final signal value ofthe pixel of interest.
 8. A non-transitory computer-readable recordingmedium storing a program for causing a computer to function as the imageprocessing apparatus according to claim 1.